I am surprised at how much children remember too.
I was quite the nazi-big sister back then. Now my
siblings, sister especially, tell me how traumatized she was when
under my care. She can recall me dishing out punishment and
being a stern teacher etc and says that I've scarred her for
life :P
Sounds almost like borderline abuse although I maintain till now
it's just tough love LOL I can't recall all of the incidents
she mention. Which proves that ometimes, adults don't
realise the kind of effect they have over children
(overly-sensitive or not is another matter).
When mom and I had one of our heart-to-heart talks a few years
ago, she told me that I was a very happy and darling toddler that
was very popular with the neighbours and relatives, until my
younger bro popped out. After that, I became an absolute
terror, either endlessly seeking attention, or withdrawing into
some corner away from everyone who's inevitably doting on the new
baby.
I was a Mr Jekyll and Dr Hyde when it came to my bro. I
had my Nazi-Sister from Hell moments where pure jealousy turned
into rage and yeah... borderline abuse.
There were also moments where I loved my bro to death and wanted to
protect him and watch over him.
Yeah, my mom didn't realise the impact of her actions on
me. She was stunned beyond words.
I remember very vividly one night, sitting at the kitchen table
slaving over my Primary 2 English workbook, asking my mom, "Mom,
how do you spell 'Giraffe'?"
Mom, after a hard day at work, muttered, "G-i-r-a-f-f-e!"
impatiently and too quickly. I asked her again, she got more
impatient. I held my pencil, stared at the red rubber tip,
still didn't catch, gulped and decided to ask her a 3rd time...
SLAP!!! 

I still remember how that stung! The heat rushed to my
cheek, and the hot tears gushed out but I just quietly sobbed,
pencil still being held upright. She told me I was 'so
stupid, 'Giraffe' also don't know how to spell', 'jiak liao bi',
etc etc.
Primary 2... just eight years old, but till today, I
remember.
I asked her, "Did you remember hitting me over the word,
'Giraffe'?" and she went, "HUH?!" 
To her, it was just an outburst because she was tired that night
so her temper was short. To me, it was nothing short of
traumatic.